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Abstract. Wikis have been recently promoted as tools that foster collaborative learning. However, 
there has been little research devoted to the criteria that are suitable to address issues pertinent to 
collaborative learning. This paper proposes a set of criteria to explore technical and pedagogical 
usability issues of collaborative learning with wikis. The criteria are then used to evaluate students’ 
collaborative writing activities. The units of study are wikis that groups of students developed 
collaboratively using MediaWiki. This paper also reports on technical and pedagogical implications 
for the use of wikis as collaborative learning tools in teacher education. 
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1. Introduction 

Wikis can provide teachers with opportunities for creating socially engaged tasks, such as 
using wikis as a tool to track a group project, to collect data for a class project. Teachers 
can also use wikis for collaborative curriculum design and course content authoring (Le¬ 
ung and Wah Chu, 2009; Matthew and Callaway, 2009; Mindel and Verma, 2006; Parker 
and Chao, 2007). As for any digital technology, technical usability is a self-evident re¬ 
quirement for wikis. However, the added value of wikis in terms of collaborative learning 
will come from a better understanding of pedagogical issues and not automatically from 
improved technology (Govindasamy, 2002; Hamid, 2002; Watson, 2001). Hence, the use 
of wikis in teacher education must include pedagogical issues that are pertinent to col¬ 
laborative learning. The main goal of this work is to report on a study that investigates 
technical and pedagogical usability issues of collaborative learning with wikis. This paper 
is a substantially extended and improved version of published short articles (Hadjerrouit, 
2011; Hadjerrouit, 2012). 

The paper is structured as follows. First, the research questions are presented. Sec¬ 
ond, a literature review is undertaken. Then, collaboration concepts are described. This 
is followed by a set of technical and pedagogical usability criteria that are pertinent to 
collaborative learning with wikis. The article continues with the research methodology. 
Then, the results are presented. Finally, the paper ends with a discussion of the results 
and implications for the use of wikis in teacher education. 
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2. Research Questions 

The research goal of this work is to explore technical and pedagogical usability issues of 
collaborative learning with wikis. The work focuses on the following three interrelated 
research questions: 

What are the technical and pedagogical usability criteria that are pertinent to collabo¬ 
rative learning with wikis? 

How do students perform collaborative writing activities using MediaWiki on the ba¬ 
sis of those criteria? 

What are the implications that can be drawn for the use of wikis as collaborative 
learning tools in teacher education? 


3. Literature Review 

Technically, wikis have been defined as Web-based systems that are easy to use and asyn¬ 
chronous, allowing users to develop content collaboratively from the bottom-up (Hughes 
and Narayan, 2009; Lamb, 2004). The core properties of wikis are: ease-of-use, simple 
navigation, linking features, and user involvement (Tetard et al., 2009). From a pedagog¬ 
ical point of view, wikis have been promoted as tools that foster collaborative learning 
and writing, knowledge sharing, group discussion, and active participation among stu¬ 
dents (Norten and Hathaway, 2008; Parker and Chao, 2007) 

The literature review identifies a number of problems associated with wikis. For ex¬ 
ample, Minocha and Thomas (2007) pointed out that is there is a need to support the 
discussion aspects of collaborative activities with more appropriate tools. Another prob¬ 
lem with wikis is that they allow a participant to edit the content of a Web site. This raises 
questions of copyright, because students may use others’ work as their own (Heafner and 
Friedman, 2009). Also the problem of concurrent updating is a recurrent problem that 
creates frustrations among students. 

Furthermore, a number of research studies seem to confirm that students appear to 
favour individual work over collaboration using wikis, and that limited student contribu¬ 
tion to the wiki seems to be a serious problem. For example, Elgort et al. (2008) indicated 
that a significant number of students thought that they could have done the task better 
on their own without collaborating with others. Likewise, Ma and Yuen (2008) pointed 
out that only half of the students were satisfied with the use of a wiki for collaborative 
writing. Carr et al. (2007) concluded that some students are reluctant to use wikis for 
online course work. Moreover, Cole (2009) reported that after five weeks, the students 
had not contributed to the wiki at all. Similarly, Ebner et al. (2008) indicated that none 
of the 287 students involved in wiki created new pages or edited existing pages over the 
whole semester. These examples show that the use of wiki does not automatically en¬ 
sure collaboration, and that additional elements, or even a new pedagogy, are required to 
promote collaborative learning. Karasavvidis (2010) argues that most problems experi¬ 
enced by the students with wikis hint at a “fundamental problem, namely the dominant 
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traditional practices and the associated learning epistemology which is compatible by 
such practices” (p. 226). Leung and Chu (2009) also pointed out that wiki in education is 
associated with a careful consideration given to a sound pedagogy. 

Another issue that emerged from the use of wikis for collaboration is the problem of 
ownership. Wheeler et al. (2008) reported that students tend to protect their ideas, and 
are resistant to having their contributions to the wiki changed by other group members. 
Likewise, Arnold et al. (2009) pointed out that students were more concerned with adding 
content to the wiki, and rarely revise the content edited by peers. Similarly, Lund and 
Smprdal (2006) found that learners do not like to rewrite or edit their own or peers’ 
contributions to the wiki, or interact with others’ material. There are also reports of the 
unwillingness of students to engage effectively in collaboration, because they do not want 
to change or modify others’ work (Britcliffe and Walker, 2007). Furthermore, Minocha 
and Thomas (2007) reported that students did not mind critiquing others’ work, but the 
nature of the critical reviews was not perceived as being positive by some students. 

Finally, Forte and Bruckman (2006, cited in Judd et al., 2010) reported that students 
tended to postpone the most important part of the work close to the project deadline, 
while less significant contributions to the wiki were spread out over a longer period of 
time. Similarly, Carr et al. (2007) pointed out that much of the work still fall to few 
students even when the participation is relatively high. 


4. Collaboration Concepts 

4.1. Collaboration vs. Cooperation 

Cooperation is seen as work where participants split a task in subtasks among them¬ 
selves, and work independently from each other (Ta-Elhasid and Meishar-Tal, 2007). In 
contrast, participants collaborate when they co-ordinate their efforts to accomplish a task 
collectively (Ta-Elhasid and Meishar-Tal, 2007; Resta and Laferriere, 2007). In educa¬ 
tion, cooperation is defined as the division of work between students who are faced with 
a joint activity, while collaboration involves the “mutual engagement of participants in a 
coordinated effort to solve the problem” (Dillenbourg et al., 1996, p. 190, cited in Judd 
et al., 2010). 

4.2. Collaborative Learning 

Collaborative learning describes a learning process generated by groups of students work¬ 
ing together as a team. Through collaboration students contribute to each others’ learning, 
and thereby create a social learning environment that is more fruitful than the addition of 
individual work (Thompson and Ku, 2006). Witney and Smallbone reported that groups 
of students “can achieve more collectively than individuals” (p. 102). Collaborative learn¬ 
ing is grounded in Vygotsky’s socio-constructivist learning theory, which assumes that 
learning occurs through collaboration and information sharing in authentic contexts. Vy¬ 
gotsky (1978) argued that the way students learn is shaped by their relationships with 
others. 
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4.3. Collaborative Writing 

Collaborative writing is an activity that transforms a text by multiple students into a col¬ 
lective document (Bradley et al., 2010; Chao and Lo, 2009; Tal-Elhasid and Meishar-Tal, 
2007; Trentin, 2009). The benefits of collaborative writing are viewed as very positive for 
the learning process. According to McConnell (2005, cited in Bradley et al., 2010), col¬ 
laborative writing fosters critical thinking, helps to clarify, explain, and reflect on ideas, 
enables information sharing, and develops discussion and communication skills. Collabo¬ 
rative writing also offers opportunities to practice literature review, academic reading and 
writing (Kim et al, 2009; Trentin, 2008). Yarrow and Topping (2001) claim that students 
working collaboratively improved significantly more than those who wrote alone. 

4.4. MediaWiki as Collaborative Learning Tool 

MediaWiki is seen as a tool that supports collaborative learning and writing (Kasemvilas 
and Olfman, 2009; Wheeler et al., 2008). As such, MediaWiki is supposed to stimulate 
students to share knowledge, discuss issues of common interest, participate, and work 
together to reach a common goal (Chu et al., 2009; Hughes and Narayan 2009; Jones, 
2010). MediaWiki provides functionality to carry out collaborative writing activities, e.g., 
editing content, tracking students’ contributions, and comparing the differences between 
two versions of the wiki. Teacher can use these features to assess the distribution of work 
among students and the types of activities carried out on the wiki. MediaWiki has also a 
page that supports a discussion forum and written communication. 


5. Technical and Pedagogical Usability Issues of Wikis 

As any other digital learning technology, wikis are associated with technical and peda¬ 
gogical usability factors that influence the way students perform collaborative writing ac¬ 
tivities. The research literature provides a set of criteria that can be used to assess the tech¬ 
nical and pedagogical usability of wikis (Hazari et al, 2008; Leacok and Nesbit, 2007; 
Nielsen, 2000; Nokelainen, 2006). Basically, technical usability involves techniques for 
ensuring a trouble-free interaction with the software while pedagogical usability aims at 
supporting the learning process (Hadjerrouit, 2010). Hence, the goal of technical usabil¬ 
ity is to minimize the cognitive load resulting from the interaction with the wiki in order 
to free more resources for collaborative learning. Yet, technical and pedagogical usability 
cannot be considered as separate, disjointed activities. 

5.1. Technical Usability Criteria 

Nielsen’s factors of Web usability (Nielsen, 2000) can be adapted to wikis with slight 
modifications. These include content, page, and site design. Page design describes how 
easy it is to read Web pages. Content design depends on writing for scannability and 
media use. Site design is about linking and navigation. In educational settings, technical 
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usability expresses how a web site is convenient, practicable, and usable for the students. 
When applied to wikis, technical usability is divided into four criteria: 

• Ease-of-use. This criterion indicates how easy it is to read the content of the wiki 
and associated figures, images, and illustrations. 

• Efficiency. This criterion describes the extent to which time and efforts are well 
used for the intended wiki task or purpose. 

• Technical design. This criterion expresses the quality of the wiki design in terms 
of page structure, placing of images, tables, and illustrations. 

• Accessibility and navigability. This criterion specifies how easy it is to access the 
wiki, and navigate through its pages. 

5.2. Pedagogical Usability Criteria 

To capture pedagogical issues that are fundamental to collaborative learning, the usability 
concept needs to be extended beyond technical usability. Nokelainen (2006) defined a set 
of pedagogical criteria that can be applied to any digital learning material, such as learner 
control, collaborative learning, added value, motivation, flexibility, feedback, etc. These 
can easily be adapted to wikis with slight modifications according to their specificities. 
Hence, to capture the very essence of wikis, pedagogical usability is broken down into a 
set of specific criteria: 

• Added value. The primary added value of wikis lies in their potential power to 
support collaborative learning and writing in comparison to traditional technologies 
such as LMS, CMS, or text processing systems. The added value of wikis also 
lies in their openness, ease-of-use, discussion forum, and assessment of students’ 
contributions to the wiki. 

• Motivation. Motivation is an important factor affecting learning. Two types of 
motivation can be distinguished: internal and external motivation (Anderman and 
Dawson, 2011). Internal motivation is a function of the value placed on the wiki, 
and the amount of efforts a student is willing to invest in working with it. The 
motivation increases when the wiki is inherently enjoyable and contain intrinsically 
information that has a highly value for the student. External motivation refers to 
motivation that comes from outside a student performing wiki tasks in order to 
benefit from them, e.g., passing an exam. 

• Differentiation. This criterion is used to adapt wikis to the users’ needs. It involves 
fitting the wiki to the characteristics of the users, e.g., age, gender, preferences, 
language, and prior knowledge (Hadjerrouit, 2010). Differentiation is important to 
attract potential users interested in the wiki. Knowing that there might be an audi¬ 
ence for their wiki motives students to develop well-structured wiki pages using a 
clear and understandable language, 

• Collaboration. The very nature of wikis lies in their potentialities to support col¬ 
laboration among participants. True collaboration requires one student to modify 
the content posted by another student and re-working the writing of others. In con¬ 
trast, collaboration may occur at a lower level, when a student simply adds content 
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to an existing wiki page (Mindel and Verma, 2006). Genuine collaboration requires 
that all participants contribute to all aspects of the wiki application: content, struc¬ 
ture and language (Chao and Lo, 2009). 

• Discussion. This criterion describes the way and the extent to which the wiki is 
used for discussion and communication. Basically, the wiki discussion page is used 
as a space for communication among participants. It can be used to discuss different 
aspects of the wiki tasks. It may also be used to transform and improve the tool to 
a better instrument for collaboration and coordination, for example adding the date 
of contribution and name of contributor. 

• Assessment. The assessment criterion is important for evaluating students’ contri¬ 
butions to the wiki. Of particular interest is the data log of MediaWiki that tracks 
activity and stores previous versions of the wikis by observing who is active, and 
when, the type of activities, etc. The log permits the assessment of students’ con¬ 
tributions in terms of collaborative activities performed on the wiki, level of con¬ 
tribution, timing and work intervals. 

• Peer-review and feedback. In addition to collaborative activities within their own 
group, students need to benefit from comments and feedback received from other 
groups. Peer review needs to be well-organized and structured in terms of content 
and issues in order to be beneficial to the students. Peer review is also important 
from the point of view of academic writing. 


6. Methodology 

This work uses a case study to examine the technical and pedagogical benefits and chal¬ 
lenges of wikis to support collaborative learning. The units of study were wikis that group 
of students developed collaboratively using MediaWiki. 

6.1. Participants 

Participants were 9 students enrolled in a course in Web 2.0 technologies offered by 
the University. None of the students were involved in wiki projects, or had pre-requisite 
knowledge in collaborative learning and writing. Three groups of students were involved 
in threewiki projects associated with collaborative writing. The teacher was not supposed 
to restrictively control the wiki information and the writing methods the students prefer 
to adopt, and the way they collaborate. 

6.2. Learning Tasks 

The wiki tasks need to fulfil both technical and pedagogical usability requirements. First, 
the wikis should be at least 4000 words in length. The wiki pages can vary in scope, but 
some pages need to provide more than rudimentary information. Some pages may contain 
suggestions of good quality that provide the seed for a more thorough article. 
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Second, the wikis should have a good technical design. They need to be in accordance 
with guidelines for writing good articles, layout, editing, style, and use of references. 
Students should usebasic wiki functions to edit content, upload hies and images, discuss 
issues associated with the wiki topics, and review others’ wikis. Third, the wikis should be 
self-explaining and adapted to the users’ needs, and offer information of good quality. The 
wiki projects must contain a description of the goal of the wikis and the target audience. 

6.3. Wiki Projects 

The subjects of the wiki projects were chosen by the students in collaboration with the 
teacher. The objectives and topics of the wiki projects are: 

• Predators. The purpose of this wiki is to present information about predators in a 
straightforward manner so that one can find information about the animals without 
having to spend much time. The target audience are all groups of people, including 
school students, who want to learn more about predators. 

• Planets. This is a topic that is very relevant to school students in lower secondary 
schools. It is important for this group of users to understand the specificities of the 
eight planets of the sun system. The main goal is to collect and structure informa¬ 
tion, images, and illustrations about the planets. 

• Nutrition and diet. This wiki is intended as an aid to school students in the 9th 
grade in lower secondary schools. The purpose is to help school students find facts 
about various energy sources regarding diet and nutrition according to the recom¬ 
mendations of the Social and Health Directorate, 

The wikis were developed by three teams of students in collaboration with the uni¬ 
versity teacher, who provided supervision, on the one hand, and fellow students who 
provided peer review, on the other hand. The students’ contributions to the wiki projects 
were not assessed individually, but as a group work. 

6.4. Procedure 

The group projects lasted for eight weeks, and were divided into five stages. At the 
end, the students delivered the final wiki products they developed using MediaWiki. The 
instructor provided advice throughout the development process and good examples of 
wikis. The five stages that the students followed were: 

1. Selection a wiki topic and collection of information about the topic 

2. Development of the wiki and associated pages 

3. Peer review and feedback from fellow students using a survey questionnaire with 
open ended questions. 

4. Self-evaluation of technical and pedagogical issues of the wiki using a survey ques¬ 
tionnaire with open ended questions 

5. Delivery of the final wiki product 
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6.5. Data Collection and Analysis Methods 

To investigate technical and pedagogical usability issues of wikis, particular attention was 
devoted to a combination of different collection and analysis methods: 

• students’ self-evaluation using a questionnaire with open-ended questions; 

• students’ peer review based on a questionnaire with open-ended questions; 

• students’ discussion log in the respective wikis; 

• students’ contributions to the wiki in the data log of MediaWiki; 

• informal discussion with the students over the duration of the wiki projects. 
Regarding the first method, students were asked to express their perceptions about 

technical and pedagogical issues of wikis by filling a survey questionnaire with open 
ended questions so that they can comment their choices, preferences, and evaluations, as 
well as what they liked or disliked with the wiki, and what should be improved. 

Peer review is a method used for evaluating the work performed by peers to ensure 
it meets specific criteria. The goal of peer review is to judge whether the wiki satisfies 
specific criteria, and provide suggestions for improvements. As for self-evaluation, peer 
review used a questionnaire with open ended questions. The results of self-evaluation and 
peer review were analysed and interpreted qualitatively. 

The discussion log is used to evaluate the extent to which students discussed issues 
related to the wikis. The content of the log is interpreted qualitatively. 

The data log allows a quantitative analysis of the students’ contributions to the wikis 
made by each member of the groups. 

Informal discussions were used as a supplementary method to provide more informa¬ 
tion about the students’ perceptions of the wikis. 


7. Findings 

7.1. Technical Usability 

7.1.1. Ease-of-Use and Efficiency 

Most students agreed that MediaWiki was easy to use. The ease-of-use of the tool may be 
explained by the fact that the students did not need to acquire knowledge about HTML 
and editing skills as this student clearly posted: 

... Because you do not need to have HTML knowledge to edit, or add content. If 
you wish, every thing can be done using the rich-text editing. The treatment of the 
text was easy, even though the wiki lacked a number of templates and extensions. 

In contrast to the ease-of-use, most students indicated that MediaWiki was of average 
quality regarding the efficiency criterion. It appears that the instability of MediaWiki 
strongly impacted the efficiency of the tool and the way the wikis were developed. It 
was then not surprising that efficiency problems created frustration as this student clearly 
indicated: 
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... It was very frustrating when the server dropped all the time , especially when 
we just sat and wrote. It meant that we did not find out that the server was down 
before we pressed the "Save” button and then lost everything we had edited (...). 
It was not so easy when not all the features worked as they should because the 
wiki installation was not complete (.. .). What makes it even more frustrating is 
that we could not do anything about it if anything happened with the wiki server. 
If we had full access and permissions, we could at least try to solve the problems 
that arose, instead of being dependent on a teacher (...) should fix it (...). 


7.1.2. Technical Design, Accessibility, and Navigability 

Less than the majority of students indicated that the technical design is of average qual¬ 
ity. This may be explained by the lack of extensions and templates. Furthermore, most 
students indicated that it was easy to navigate through the wiki, and that the number of 
images was appropriate. Finally, the majority of the students pointed out that the content 
was understandable. However, there was no evidence in the discussion log that the wiki 
content was critically discussed. The discussion comments focused more on technical 
aspects rather than issues of the subject matter. 

7.2. Pedagogical Usability 
7.2.1. Added Value 

Surprisingly, most students indicated that MediaWiki was not easier to use than Word, 
even though MediaWiki has more intuitive menu choices and technical opportunities. 
One of the reasons it that Word is a program that students have used for years. So even 
if MediaWiki has a simple user interface, it cannot be said that it is easier than Word. 
Some students also believed that they can do exactly the same with Word as with Medi¬ 
aWiki, and perhaps much easier and faster. They did not feel dependent on MediaWiki 
for it to work. Furthermore, one student reported that he/she preferred to use Word before 
she/he pasted the text into the wiki, often because it was easier to keep track of what she 
wrote on a document and because of the ability to save unfinished products without being 
’’viewed”. Lack of experience with MediaWiki was also a reason to prefer Word as this 
student indicated: 

. .. Because I am used to Word. I have never created a wiki article before, and it 
is both faster and more efficient for me to open a Word document than to create a 
wiki page. 

In addition to students’ preferences, it appears that technical problems prevented stu¬ 
dents from being aware of the potentialities provided by MediaWiki: 

... Had MediaWiki been installed 100% from the start we had done things a little 
differently with reflist, footnotes, etc. 

... Naturally, problems with the wiki, which hindered the use of the discussion 
forum (...), resulted in face-to-face communication or using alternative solutions. 
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7.2.2. Collaboration and Discussion 

Most students indicated that they liked to comment and edit each others’ contributions, 
that is add, delete, modify, and format content. In stark contrast, they believed that collab¬ 
oration did not increase substantially, and that collaboration was rather of average quality. 
The majority of the students also indicated that MediaWiki does not foster collaboration 
among participants. There are many reasons that may explain their responses. 

First, most students indicated that oral and face-to-face discussions were equally im¬ 
portant as online communication. There were several responses of which the following 
are representative: 

... I think the whole discussion tool was unnecessary and redundant. (...). Oral 
discussions are better alternatives to online discussion ways (...). It is also very 
redundant to engage and comment on the wiki when we more or less always sit 
together and work. 

. .. Because there was a lot of down time(.. .), we were forced to use other forums 
to discuss and edit the material so that we can achieve better consistency. 

.. . As mentioned above, such tasks were more or less accomplished through other 
forums (...) due to the unstable wiki page. 

In addition to face-to-face discussions, the students pointed out that they needed to 
use other communication channels: 

... The answer is yes and no here; this because it is easy to communicate today 
over other types of channels such as IM (Instant Message), Skype and similar 
tools. 

... One feels then that the discussion tool has the potential to be used sparingly 
when it is easier to send the message, call and get an answer right away, enter the 
mail there and wait. 

... We have not taken advantage of the discussion forum. The reason is that we as 
a group have largely worked together. It is easier to talk to the person sitting next 
to you than to write down a message using the tool. 

... Yes, I think, but it is not based on the discussion forum of the wiki. How much 
time is spent online does not need to reflect how the actual group work has been. 
I feel that my group have had a very good collaboration, but it does not look like 
when you take a look at the discussion forum board. 

Furthermore, the limited capacity of MediaWiki does not facilitate multiple-editing, 
because the tool was unable to cope with simultaneous editing. Students reported their 
disappointment: 

... It happened to me several times when someone else in the group did something 
at the same time in the wiki, edited and used the ’’Save ” button before me, then I 
lost everything I had edited, and none of my changes were saved. (...) If a word 
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was wrong or a paragraph should be somewhere else, it was easy to do this. The 
problem is that it does not happen in real time as Google Docs (...). 

The students mentioned a number of technical and pedagogical reasons to explain 
why MediaWiki did not substantially increase collaboration: 

... I cannot see that the discussion forum of the wiki raises the level of collabora¬ 
tion. It would be the same quality without. 

... Collaboration did not increase, but it was a very straightforward kind of col¬ 
laboration, in addition to other forums. 

... MediaWiki has nothing to do with it when we all sit together to work and talk 
face to face continuously. 

. .. MediaWiki made us very frustrated, and this forced us to work even more 
closely than we already did. This enhances the group, but it was not necessarily 
positive. 

... I think that MediaWiki promotes group collaboration, but because of the unsta¬ 
ble server, we have worked together in ways other than the discussion forum. We 
used email and Google Talk. But we have obviously tested the discussion forum 
and understood well the advantages of using it. 

The lack of familiarity with MediaWiki also was also an obstacle for increased col¬ 
laboration: 

... It took some time to become familiar with special wiki tags such as tables, links, 
etc. When this was put in place, MediaWiki becomes a good tool for collaboration. 

Finally, another explanation was that collaboration with MediaWiki does not happen 
in real time as with Google Docs, which was considered to be inherently superior than 
MediaWiki regarding concurrent writing: 

... It is not effective, because things do not happen in real time as in Google Docs. 


7.2.3. Motivation 

The majority of the students felt that they were motivated to use wikis, but they also gave 
both technical and pedagogical explanations for low motivation, e.g.: 

. .. Due to all the problems this task has brought with it, the motivation was rather 
negative for further work. 

. .. Personally, I feel that the task is too focused on content and not enough on how 
to use a wiki to its full potential. 

... It is motivating when the collaboration in the group runs smoothly. 

. .. Because the wiki did not work as it should. The wiki could have been written 
as a report in Microsoft Office Word. 
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7.2.4. Differentiation 

Students pointed out that the wikis were motivating and interesting to work with, be¬ 
cause of their intrinsic value for them and the users. Most students believed that the wikis 
are tailored to the needs of the users through an appropriate balance of page, site, pre¬ 
sentation, and navigation design, and that the wikis are tied to the users’ needs, age, and 
interests. They also indicated that the technical design was good and adapted to the users’ 
preferences regarding colours, links, and navigation, although the wikis lacked multime¬ 
dia elements. Furthermore, they reported that the amount of text and figures provided 
by the wikis is suitable, and this helped the users to learn more. Finally, it seems that 
most students were aware of the fact that wikis cannot always cover the entire knowledge 
level of the users. It is therefore important for the learning process to combine wikis with 
textbooks. 

7.2.5. Assessment 

Central element for the assessment of students’ contributions was the data log, which 
provides all interactions with the wikis, chronologically listed, displayed date, authors 
and changes made in the text. The log is particularly useful to support data analysis be¬ 
cause it kept track of the individual contributions to the wikis made by each member of 
the groups. A detailed analysis of the data log shows that group members mostly worked 
on individual sections of the wikis. There were only few occasions when the students 
revised substantially each other’s work. Then, the assessment of the data log revealed 
that the contributions were not evenly distributed in two groups. The result was that one 
or two students performed most of the wiki tasks. It also happened that some students 
copied items from other sources, and pasted them directly into the wikis. The students 
learned to write their own commentaries after the teacher made them aware of this prob¬ 
lem. Moreover, it appears that all groups worked much as the last deadline approached, 
and did not follow the schedule assigned throughout the project period, as the increased 
number of activities during the last weeks clearly shows. This reduced the possibility of 
collaborative writing further. 

7.2.6. Peer Review and Feedback 

Peer review was carried out three weeks before the end of the wiki projects. At this 
point of time, the wikis lacked a number of features. However, it was important to review 
the students’ tasks since these needed to be improved to achieve a better quality of the 
final products. Peer review revealed that students basically agreed that the wikis were of 
average quality from the technical point of view. Efficiency was one of the most important 
problems that the students mentioned as these comments reveal: 

... A complete MediaWiki installation with all features and the sender stability. 

. .. Additional templates/extensions should be installed, or alternatively, include 

your wiki in a project to avoid unnecessary work in the form of tasks that do not 

‘count’ in the final grad. 
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... Efficiency has necessarily nothing to do directly with MediaWiki, but should 
at least be addressed so that any further use does not have to work with word 
processors to copy and paste into the wiki. 

Furthermore, some students were disappointed that MediaWiki lacked some advanced 
extensions and functions, even though these were not required: 

... MediaWiki is ok for its purpose, but for this task we might as well have written 
a report with Microsoft Office Word. The MediaWiki installation is missing a lot of 
settings and functions so that for example it is not possible to use <reflist> cqoute 
feature or function, even if there is a requirement for the task. 

... I get the impression that the installation is not complete, at least that some 
features are missing. This is something that I think is significant, whether or not 
there are functions that are ”required” in the task or not. When I first must learn 
about MediaWiki I want to learn everything, and not just what is written in the 
project task. 

Regarding the degree of collaboration, the majority of the students believed that this 
cannot be characterized as high, and that MediaWiki did not automatically foster col¬ 
laboration, because the discussion forum was not good enough to promote collaborative 
learning. 

... The discussion forum shows that the group has worked together, but the forum 
is not used actively. Part of the reason may be that the group has always worked 
together. The discussion forum has not been necessary. 

... The discussion channel is used, but I doubt that collaboration will be promoted 
any more than if it had not been used. This is because the group usually gathered 
when we were working, and an oral discussion is better and easier. 

... It does not seem like there is so much discussion or people who have edited, 
but due to occasional problems with the wiki we can certainly assume that it has 
been discussed in other forums. 

... Low academic discussion, most ’’Facebook speak’’. 

Concerning the differentiation criterion, most students believed that the wiki topics 
are suited to the users’ needs, because they contain relevant information that is potentially 
interesting and motivating. 


8. Discussion 

Wikis have been promoted as a new technology that fosters collaborative learning. How¬ 
ever, in spite of these potentialities, students may be left frustrated or disappointed, be¬ 
cause wikis do not sufficiently address their needs or expectations. Both technical and 
pedagogical problems may hamper collaborative learning and writing. Even though wikis 
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are considered as user-friendly, technical problems may arise for different reasons: unsta¬ 
ble server, lack of extensions, downloading of bles, concurrent editing etc. Beyond wiki 
technicalities, pedagogical usability problems are still the main factor that impacts col¬ 
laborative learning, such as the integration of wikis into teaching and learning processes, 
students’ prior knowledge and experiences with wikis, familiarity with MediaWiki, etc. 
Also reliability and validity issues need to be addressed. Several lessons can be drawn 
from the findings for an appropriate use of wikis as collaborative learning tools in teacher 
education. 

8.1. Technical Usability Issues 

The findings indicate that wikis still lack a number of functionalities that would make 
them more usable, and that several technical usability problems need to be addressed 
before MediaWiki can be used successfully to promote collaborative learning. First, the 
discussion tool in its current form needs to be improved to help students follow a discus¬ 
sion tread. Students had to write down the date and the name of the contributor, which 
should automatically appear on the discussion page. Then, MediaWiki needs a locking 
mechanism to avoid the problems of concurrent updates. The tool should be modified so 
that a particular wiki page on which a student is currently working would be locked for 
usage. Then, it is impossible to escape network connection problems, and the frustration 
and low motivation they caused. Moreover, adding extensions and modules would be ad¬ 
vantageous for students, so that more of the MediaWiki functionality could be used to 
improve the technical quality of the wikis. Of course, many of these problems cannot be 
solved without the redesign of MediaWiki. Nevertheless, both teachers and students need 
be aware of the fact that technical usability obstacles can over-shadow the pedagogy and 
hamper the learning process. 

8.2. Pedagogical Usability Issues 

Students’ perceptions of collaborative writing by means of self-evaluation and peer re¬ 
view, and analysis of students’ discussion logs as well, reveal that collaboration and dis¬ 
cussion among students cannot be characterized as high and that MediaWiki does not 
automatically foster collaboration. These findings are confirmed by the data log, which 
shows that the students were more inclined to cooperate than collaborate to carry out 
the wiki projects. One reason for low collaboration is the lack of collaborative skills and 
familiarity with MediaWiki, since none of the students were involved in wiki projects be¬ 
fore, Another reason is that true collaboration may be a real challenge for any student as 
it is cognitively demanding. True collaboration is difficult to achieve, unless students pos¬ 
sess higher-order academic skills and critical awareness to judge the information posted 
on the wiki by other students (McLoughlin, 2007). Otherwise, students tend to accumu¬ 
late content on wiki pages as the data log of MediaWiki clearly shows. Moreover, students 
were inclined to postpone their work as the project deadline approached. This reduced the 
possibility of collaboration, in line with the research literature (Meishar-Tal and Gorsky, 
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2010). Clearly, as Cole (2009) pointed out, it is not enough to simply use wikis in courses 
without a radical change of the underlying pedagogy and learning paradigm, and expect 
students to automatically collaborate. Rather, course content and pedagogy need to be 
redesigned to realize the potential capabilities of wikis in education. 

The findings reveal that collaborative learning benefits cannot be expected as long 
as the affordances and limitations of wikis are not taken into consideration. In this re¬ 
gard, the role of the teacher cannot be underestimated. Students need to be taught that 
wikis’ potential capabilities lie in supporting a collaborative approach to learning. Fur¬ 
thermore, students need to be taught that the use of wikis may create problems connected 
to the trustworthiness of the information and the risk for plagiarism. Teachers also need 
to emphasize academic thinking, and guidelines for writing good articles (Wheeler and 
Wheeler, 2009). The role of the teacher consists of guiding the students and providing 
appropriate feedback to carry out their tasks (Nordin and Klobas, 2006). However, as fa¬ 
cilitator of collaborative writing, the teacher is not supposed to restrictively control the 
study material and the methods the students prefer to use. Instead, students should feel 
confident with methods and approaches that suit their personal learning style. 

Since students were more inclined to cooperate rather than to collaborate, it may be 
necessary to develop an awareness of the difference between cooperation and collabora¬ 
tion, and the way collaborative writing might be carried out. Clearly, putting students to¬ 
gether does not automatically result in collaborative work, partly because the acquisition 
of collaborative skills must be addressed before taking advantage of wikis. Such skills 
become necessary to foster collaborative learning. It follows that collaborative learning 
should not be restricted to wikis alone but should be possible using any means found 
useful, for example allow students come together to discuss a topic, especially when the 
participants have different backgrounds and can add to each other’s knowledge, and re¬ 
flection through co-author summaries of what they have learned (Tetard et ai, 2009). 

Furthermore, most students indicated that the online discussion page alone is not the 
best tool for communication and dialogue, because the need to engage in some form 
of synchronous communication is still highly valued by the students. It appears that the 
blended or “hybrid” model of communication is the most appropriate form of discus¬ 
sion among students, because the combination of different forms of communication are 
more stimulating for the learning process than one tool alone can do (Weber, 2008). 
The blended model involves both synchronous and asynchronous communication. Syn¬ 
chronous communication is based on face-to-face meetings, supplemented by traditional 
and new forms of asynchronous communication, such as email, and new technologies for 
social interaction such as Google Docs, and Instant Messaging. Hence, all forms of com¬ 
munication and collaborative technologies are potentially valuable to groups of students 
collaboratively working to develop wikis. 

In addition, students need to be aware of the role of motivation, since it is an es¬ 
sential component of collaborative learning. Motivation can be achieved in many ways. 
First of all, the motivational value of wikis must be seen in relation to the wiki topic it¬ 
self, whether it is intrinsically interesting and highly relevant to the students. Indeed, the 
data log shows that motivated students edited more content and used more wiki functions 
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than other students. A number of factors may explain low motivation: lack of collabora¬ 
tive skills, technical problems with MediaWiki, lack of clear assessment procedures, etc. 
Motivation can be achieved in many other ways, for example through wikis that allow 
students to take some control over their own learning, or wikis that provide opportunities 
for high-level group activities and encourage learner participation (Leacock and Nesbit, 
2007). High motivation can also be achieved through performance goals, such as passing 
the course (Nokelainen, 2006). Finally, multimedia possibilities afforded by the wikis can 
increase student motivation. 

Furthermore, there is no clear evidence that the way the students were assessed, as a 
group and not individually, influenced the methods they used to develop wikis. Neverthe¬ 
less, assessment plays an important role in evaluating the students’ contributions, and can 
enhance student achievement. MediaWiki has a data log that keeps a record of students’ 
individual contributions to the wiki. The log enables a proper follow-up of the students’ 
edits, which in turn allows a quantitative evaluation of the contributors’ individual in¬ 
put. From the teacher’s point of view, an assessment of students’ individual contributions 
must be made, even though this evaluation form could be a sensitive issue for students 
working together to achieve a common goal. However, quantitative assessment does not 
need to be the only way to evaluate students’ individual contributions, since statistical 
data alone give little information about the quality of the input. Other assessment ways 
may use other tools than the wiki data log, such as self-assessment and peer assessment, 
on an individual basis or in groups (Tal-Elhasid and Meisjar-Tal, 2007). 

Finally, the benefit of peer review cannot be underestimated (Rieber, 2006). Dur¬ 
ing peer review, students evaluate other students’ wikis to find out if they followed the 
projects requirements. This process gives students a possibility to look at the requirements 
once again, because they are assessing whether other students followed them. As a result, 
they may be especially careful to reassess their own understanding of the wiki tasks. In 
turn, a careful understanding of the tasks may help the students revise their writings and 
improve the wikis after the peer review process. 

8.3. Reliability and Validity Issues 

Reliability is concerned with the question of whether research findings are repeatable 
and consistent over time (Bryman, 2004). Hence, to achieve a high degree of reliability, 
it is crucial to be aware of the conditions under which this study was carried out and the 
factors that may impact the findings of the study, such as students’ prerequisite knowledge 
in collaborative learning and writing, prior experiences with wikis, server connections, 
lack of MediaWiki extensions, and other pedagogical, technical, and institutional factors. 
Reliability is also enhanced by a clear description of the procedure being used to carry 
out the main steps of the study so that it can be reused to produce similar results. 

This work also raises external validity issues that impact the generalizability of the 
work beyond the specific research context (Bryman, 2004). This work was conducted 
with a small convenience sample, with participants from one course only. As a result, the 
work is not representative for a larger population of students, and cannot be generalized 
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to other students, even though the results are consistent with some current research work 
(Carr et al., 2007; Cole, 2009; Elgort et al., 2008; Lund and Sm0rdal, 2006). Hence, 
the findings need to be considered with caution, until replication studies with a larger 
population confirm the results. 


9. Conclusions and Future Research 

The research objective of this article was to use a set of criteria to evaluate technical and 
pedagogical usability issues of collaborative learning with wikis. The experiences that 
have been reported in this paper demonstrate that the use of a new technology that opens 
for collaborative learning and writing can never be easy or straightforward (Vratulis and 
Dobson, 2009). The findings reveal that both technical and pedagogical issues need to 
be addressed in order to promote wikis as collaborative learning tools. Besides technical 
usability, which is a self-evident requirement, there is a need for a pedagogical approach 
that provides students with a genuine collaborative leaning model in teacher education. 
Future work will focus on the refinement of the usability criteria and the instruments 
for assessing students’ perceptions of collaborative writing activities. In addition, future 
research will be undertaken with larger student groups to ensure more reliability and 
validity. 
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Mokymosi bendradarbiaujant naudojantis vikiu techniniu 
ir pedagoginip aspektu tyrimas 

Said HADJERROUIT 

Vikis visai neseniai buvo reklamuojamas kaip priemone, galinti skatinti mokymosi bendradar¬ 
biaujant. Taciau buvo nedaug mokslinii} tyrimp apie sios priemones tinkamumo spr^sti klausimus, 
susijusius su mokymusi bendradarbiaujant kriterijus. Siame straipsnyje pateiktas siam tikslui skirtp 
kriterijp rinkinys. Sie kriterijai buvo panaudoti siekiant jvertinti besimokancipjp bendradarbiavi- 
mo rastu veiklp. Nagrinejami vikiai, kuriuos besimokancipjp grupes sukure bendrai naudodamiesi 
„MediaWiki“. Straipsnyje taip pat rasoma apie vikio, kaip mokymosi bendradarbiaujant rengiant 
mokytojus priemones, naudojimo technin§ ir pedagogin? reiksm?. 



